Moving image recording method and information processing device

ABSTRACT

A video information item obtained by photographing and an audio information item obtained by recording are digitized to obtain a digitized video data item and a digitized audio data item each accompanied by a time information item, respectively. The digitized video data item is stored in such a manner as to be separated into the time information item and the digitized video data item. The digitized audio data item is stored in such a manner as to be separated into the time information item and the digitized audio data item. The separately-stored time information item and the digitized video data item and the separately-stored time information item and the digitized audio data item are combined in synchronization with each other. The combined time information items, the digitized video data item, and the digitized audio data item are stored in a predetermined format.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a U.S. continuation application filed under 35U.S.C. 111(a) claiming benefit under 35 U.S.C. 120 and 365(c) of PCTapplication JP2007/057259, filed on Mar. 30, 2007, the entire contentsof which are incorporated herein by reference.

FIELD

The embodiment discussed herein is related to a moving image recordingmethod and an information processing device.

BACKGROUND

Up until now, there have been discussed reproducing systems capable ofseparately recording an audio data file and a moving image data file andreproducing audio and moving image in synchronization with each othereven without any dedicated processor having high processing performance.

-   Patent Document 1: Japanese Laid-Open Patent Publication No. 7-95522

SUMMARY

According to an aspect of the present invention, there is provided amoving image recording method including digitizing a video informationitem obtained by photographing and an audio information item obtained byrecording to obtain a digitized video data item and a digitized audiodata item each accompanied by a time information item, respectively;temporarily storing the digitized video data item accompanied by thetime information item in such a manner as to separate the digitizedvideo data item accompanied by the time information item into the timeinformation item and the digitized video data item; temporarily storingthe digitized audio data item accompanied by the time information itemin such a manner as to separate the digitized audio data itemaccompanied by the time information item into the time information itemand the digitized audio data item; and combining the time informationitem and the digitized video data item that are separately stored in thetemporary storing the digitized video data item with the timeinformation item and the digitized audio data item that are separatelystored in the temporarily storing the digitized audio data item insynchronization with each other so as store the time information items,the digitized video data item, and the digitized audio data item in apredetermined format.

The object and advantages of the present invention will be realized andattained by means of the elements and combinations particularly pointedout in the claims.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and arenot restrictive of the present invention, as claimed.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a function block diagram of an information processing devicethat performs a moving image recording method according to an embodimentof the present invention;

FIG. 2A is a diagram for illustrating the flow of processing by themoving image recording method according to the embodiment of the presentinvention;

FIG. 2B is a diagram for illustrating the flow of processing by themoving image recording method according to the embodiment of the presentinvention;

FIG. 2C is a diagram for illustrating the flow of processing by themoving image recording method according to the embodiment of the presentinvention;

FIG. 3 is a diagram illustrating a state in which digitized video dataitems, time information items corresponding to the digitized video dataitems, digitized audio data items, and time information itemscorresponding to the digitized audio data items are combined with eachother through time synchronization processing and format conversionprocessing and recorded in a video/audio/time file in the moving imagerecording method according to the embodiment of the present invention;

FIG. 4A is a diagram illustrating a state in which the digitized videodata items and the time information items corresponding to the digitizedvideo data items are separately recorded in the moving image recordingmethod according to the embodiment of the present invention;

FIG. 4B is a diagram illustrating a state in which the digitized videodata items and the time information items corresponding to the digitizedvideo data items are separately recorded in the moving image recordingmethod according to the embodiment of the present invention;

FIG. 5 is an operations flowchart for illustrating the flow of the timesynchronization processing in the moving image recording methodaccording to the embodiment of the present invention;

FIG. 6 is an operations flowchart for illustrating operations related toa mode change in the information processing device illustrated in FIG.1;

FIG. 7 is a diagram for illustrating the mode change in the informationprocessing device illustrated in FIG. 1;

FIG. 8 is a diagram for illustrating the operations in a low load modein the information processing device illustrated in FIG. 1;

FIG. 9 is a diagram for illustrating the operations in a high load modein the information processing device illustrated in FIG. 1;

FIG. 10 is a diagram for illustrating the operations in a restorationmode in the information processing device illustrated in FIG. 1;

FIG. 11 is a diagram for illustrating the mode change along with achange in a processing load in the information processing deviceillustrated in FIG. 1; and

FIG. 12 is a hardware block diagram of a computer in a case where theinformation processing device illustrated in FIG. 1 is implemented bythe computer.

DESCRIPTION OF EMBODIMENTS

When moving image information items photographed by cameras and recordedby microphones are digitized and recorded, MP4 is, for example, used asa recording format.

According to MP4, digitized video data items and digitized audio dataitems are recorded in order of time, and time information itemscorresponding to the digitized video data items and the digitized audiodata items are also recorded in order of time.

Here, the time information items are composed of information itemsindicating times at which the video data items and the audio data itemswere recorded and composed of information items on the data sizes of thedata items recorded at the corresponding times.

When moving images are digitized and recorded in such a format, videoinputs from cameras are sorted at every predetermined time interval toprovide video data items accompanied by corresponding time informationitems. In addition, audio inputs from microphones are sorted at everypredetermined time interval to provide audio data items accompanied bycorresponding time information items.

The video data items and the audio data items are temporarily stored ina video buffer unit and an audio buffer unit, respectively. Then, thesedata items are read and recorded in a video/audio/time file in MP4format through time synchronization processing and format conversionprocessing.

When the moving images are reproduced from the video/audio/time file,information items on the data sizes of the video data items and theaudio data items included in the time information items corresponding totimes are read as control information items to reproduce the movingimages with time. In accordance with the control information items thusread, the video data items and the audio data items are successivelyread and reproduced by amounts equivalent to the data sizes of specifieddata items. As a result, the video data items and the audio data itemsof the moving images initially taken and recorded at the same time aresuccessively reproduced in synchronization with each other.

However, the time synchronization processing and the format conversionprocessing require a certain processing time. Therefore, when the videoinputs or the audio inputs are processed at high speed and with highquality, the synchronization processing and the format conversionprocessing are not properly processed in time, which may result in theabandonment of input data items exceeding a tolerable processing amount.

Particularly, since information processing devices such as mobile phonesrequired to be downsized and reduced in weight have limitation onenhancing data processing performance, there would be a high likelihoodof such an abandonment of input data items.

The present invention has been made in view of the above circumstancesand has an object of providing a moving image recording method capableof reliably recording moving images without abandoning input data itemseven if data items to be processed are large in amount. In addition, thepresent invention has an object of providing an information processingdevice for executing the moving image recording method.

An embodiment of the present invention is described in detail below withreference to the accompanying drawings.

FIG. 1 is a function block diagram of an information processing device,e.g., a cell phone having the configuration of executing a moving imagerecording method according to the embodiment of the present invention.

The information processing device has a camera 11 serving as a videoinput unit; a microphone 21 serving as an audio input unit; a videodigitization unit 12 that digitizes video information items output fromthe camera 11; an audio digitization unit 22 that digitizes audioinformation items output from the microphone 21; a video buffer unit 13;an audio buffer unit 23; a temporary video buffer unit 14 that stores avideo time information file 14 a and a video data file 14 b; a temporaryaudio buffer unit 24 that stores an audio time information file 24 a andan audio data file 24 b; a combining unit 25 that includes a timesynchronization processing unit 15 and a format conversion processingunit 16 (see FIGS. 2A and 2B); and a video/audio/time file storage unit26A that stores a video/audio/time file 26.

The video digitization unit 12 and the audio digitization unit 22output, for example, digitized moving image information itemsphotographed by the camera 11 and recorded by the microphone 21.

The video buffer unit 13 and the audio buffer unit 23 store digitizedvideo data items and digitized audio data items output from the videodigitization unit 12 and the audio digitization unit 22, respectively.

The combining unit 25 reads the digitized video data items and thedigitized audio data items from the video buffer unit 13 and the audiobuffer unit 23, respectively. In the combining unit 25, the timesynchronization processing unit 15 applies time synchronizationprocessing to these digitized video data items and the digitized audiodata items, and the data format processing unit 16 applies data formatprocessing to the digitized video data items and the digitized audiodata items thus subjected to the time synchronization processing,thereby generating the video/audio/time file 26. The combining unit 25stores the generated video/audio/time file 26 in the video/audio/timefile storage unit 26A serving as a recording medium.

The temporary video buffer unit 14 and the temporary audio buffer unit24 have the digitized video data items and the digitized audio dataitems temporarily stored therein in a high load mode described below.

FIG. 2A is a diagram for illustrating operations of the informationprocessing device in a low load mode described below along with FIG. 1.

According to the embodiment of the present invention, as illustrated inFIG. 2A, video inputs from the camera 11 are sorted at everypredetermined time interval to provide video data items accompanied bycorresponding time information items. In addition, audio inputs from themicrophone 21 are sorted at every predetermined time interval to provideaudio data items accompanied by corresponding time information items.Then, the video data items and the audio data items are temporarilystored in the video buffer unit 13 and the audio buffer unit 23,respectively. The video data items and the audio data items are read andrecorded in the video/audio/time file 26 in MP4 format through the timesynchronization processing unit 15 and the format conversion processingunit 16 of the combining unit 25.

The video/audio/time file 26 has a data structure as illustrated on theright side of FIG. 3. When the video/audio/time file 26 reproducesmoving images, the time information items for respective times stored ina time information section 26 a at the upper stage of thevideo/audio/time file 26, i.e., the time information items on the videodata items and the audio data items and information items on data sizesare read as control information items to reproduce the moving imageswith time. In accordance with the control information items thus read,the video data items and the audio data items of a video/audio datasection 26 b at the lower stage of the video/audio/time file 26 aresuccessively read and reproduced by amounts equivalent to the data sizesof specified data items. As a result, the video data items and the audiodata items of the moving images initially taken and recorded at the sametime are successively reproduced in synchronization with each other.

However, the time synchronization processing unit 15 and the formatconversion processing unit 16 require a certain processing time for thetime synchronization processing and the format conversion processing,respectively. Therefore, when video inputs or audio inputs are processedat high speed and with high quality, the synchronization processing unit15 and the format conversion processing unit 16 are not properlyprocessed in time, which may result in the abandonment of input dataitems exceeding a tolerable processing amount.

Particularly, since information processing devices such as mobile phonesrequired to be downsized and reduced in weight have limitations onenhancing data processing performance, there would be a high likelihoodof such an abandonment of input data items.

According to the embodiment of the present invention, as illustrated inFIG. 2B, the video information items and the audio information itemsobtained by photographing and recording are first digitized in the highload mode. Then, without being subjected to the time synchronizationprocessing by the time synchronization processing unit 15 and the formatconversion processing by the format conversion processing unit 16, thevideo information items are recorded in the temporary video buffer unit14 in such a manner as to be separated into video time information items14 a and digitized video data items 14 b, and the audio informationitems are recorded in the temporary audio buffer unit 24 in such amanner as to be separated into audio time information items 24 a anddigitized audio data items 24 b.

The processing in the high load mode does not include the timesynchronization processing and the format conversion processing, but itincludes only the process of recording the digitized data items in sucha manner as to be separated into the time information items and thevideo data items and be separated into the time information items andthe audio data items. Therefore, a required data processing amount canbe reduced. Accordingly, input data items can be reliably recordedwithout being abandoned.

As illustrated in FIG. 2C, in a restoration mode described below, thedigitized video data items 14 b and the corresponding time informationitems 14 a, and the digitized audio data items 24 b and thecorresponding time information items 24 a, which are separatelyrecorded, are successively read in order of time and subjected to thetime synchronization processing by the time synchronization processingunit 15 and the format conversion processing by the format conversionprocessing unit 16. As a result, the video/audio/time file 26 of thedata structure illustrated on the right side of FIG. 3 is obtained.

As described above, according to the embodiment of the presentinvention, when moving images are recorded in the high load mode, thedigitized video data items are temporarily stored in such a manner as tobe separated into the digitized video data items and the correspondingtime information items, while the digitized audio data items aretemporarily stored in such a manner as to be separated into thedigitized audio data items and the corresponding time information items.Then, the data items thus temporarily stored are read and rearranged inchronological order and converted into a predetermined format. Unlike arelated art in which input data items exceeding a tolerable processingamount are abandoned, the configuration according to the embodiment ofthe present invention enables the reliable storage of input data itemswithout abandoning the input data items.

In other words, according to the related art, when moving images arerecorded by, e.g., a cell phone and a microphone, digitized video dataitems, digitized audio data items, and time information itemscorresponding to the digitized video data items and the digitized audiodata items are converted into a predetermined format (e.g., MP4 format)as a single file and recorded so that videos and audios are reproducedin synchronization with each other.

In this related art, when a large amount of high quality videos areprocessed by an information processing device such as a cell phonehaving relatively low processing performance, the time synchronizationprocessing and the format conversion processing require a longprocessing time. As a result, input data items are not properlyprocessed in time, which may result in the abandonment of the input dataexceeding a tolerable processing amount.

In view of the above problem in the related art, the embodiment of thepresent invention is configured to record moving images in the low loadmode in the following manner. At first, as illustrated in FIG. 2A, videoinformation items output from the camera 11 and audio information itemsoutput from the microphone 21 are digitized by the video digitizationunit 12 and the audio digitization unit 22, respectively, as in the caseof the related art. Then, if it is determined that the storage rate ofthe video buffer unit 13 or the audio buffer unit 23 does not exceed 80%of its capacity and that the storage destinations of the present dataitems are not the temporary video buffer unit 14 and the temporary audiobuffer unit 24, the digitized video data items and the digitized audiodata items are buffered by the video buffer unit 13 that temporarilystores video data items and by the audio buffer unit 23 that temporarilystores audio data items, respectively. After that, the digitized videodata items, the digitized audio data items, which are stored in thevideo buffer unit 13 and the audio buffer unit 23, respectively, and thetime information items corresponding the digitized video data items andthe digitized audio data items are converted into a predetermined format(such as MP4 format) and stored in the video/audio/time file storageunit 26A as the video/audio/time file 26.

Here, the low load mode described above refers to a mode in which thedata amounts of the video information items input through the camera 11and the video digitization unit 12 and the data amounts of the audioinformation items input through the microphone 21 and the audiodigitization unit 22 do not exceed the performances of the timesynchronization processing by the time synchronization processing unit15 and the format conversion processing by the format conversionprocessing unit 16 of the combining unit 25. In other words, in the lowload mode, input data items are not required to be abandoned.

If the data amounts per unit time of the digitized video data itemsoutput from the camera 11 or the data amounts per unit time of thedigitized audio data items output from the microphone 21, i.e., framerates, bit rates, etc., exceed a certain value, the informationprocessing device changes to the high load mode.

Here, as a method for determining whether the data amounts per unit timeof the digitized video data items output from the camera 11 or the dataamounts per unit time of the digitized audio data items output from themicrophone 21 exceed the certain value, data amounts stored in the videobuffer unit 13 or the audio buffer unit 23 are taken into consideration.For example, if the storage data amount of a buffer memory constitutingthe video buffer unit 13 or the audio buffer unit 23 reaches 80% of thestorage capacity of the buffer memory, it can be determined that thedata amounts per unit time of the digitized video data items output fromthe camera 11 or the digitized audio data items output from themicrophone 21 exceed the certain value.

As illustrated in FIG. 2B, when moving images are recorded in the highload mode, the video information items output from the camera 11 and theaudio information items output from the microphone 21 are digitized bythe video digitization unit 12 and the audio digitization unit 22,respectively, and then separately stored in the temporary video bufferunit 14 and the temporary audio buffer unit 24, respectively. In otherwords, the digitized video information items are recorded in such amanner as to be separated into a video time information file 14 a and avideo data file 14 b, while the digitized audio information items arerecorded in such a manner as to be separated into an audio timeinformation file 24 a and an audio data file 24 b.

The video time information file 14 a, the video data file 14 b, theaudio time information file 24 a, and the audio data file 24 b aredescribed below.

In the video time information file 14 a, times at which the video dataitems were recorded and the data sizes of the video data items recordedat the corresponding times are stored. In the video data file 14 b, thevideo data items per se are stored. Furthermore, in the audio timeinformation file 24 a, times at which the audio data items were recordedand the data sizes of the audio data items recorded at the correspondingtimes are stored. In the audio data file 24 b, the audio data items perse are stored.

Here, the digitized video data items obtained by digitizing the videoinformation items output from the camera 11 and the digitized audio dataitems obtained by digitizing the audio information items output from themicrophone 21 are accompanied by time information items, i.e.,information items on times at which the information items were recordedand by information items on the data sizes of the information items inunits of data items at a predetermined time interval.

According to the embodiment of the present invention, the timeinformation items are separated from the digitized video informationitems in the high load mode. As a result, as illustrated in FIG. 4A, thedigitized video data items and the time information items correspondingto the digitized video data items are separately recorded in thetemporary video buffer unit 14 as the video data file 14 b and the videotime information file 14 a in order of time (i.e., times 0, 1, 2, . . ., N).

Similarly, the time information items are separated from the digitizedaudio information items corresponding to the predetermined timeinterval. As a result, as illustrated in FIG. 4B, the digitized audiodata items and the time information items corresponding to the digitizedaudio data items are separately recorded in the temporary audio buffer24 as the audio data file 24 b and the audio time information file 24 ain order of time (i.e., times 0, 1, 2, . . . , N).

In the high load mode, while the input data items are recorded in thetemporary video buffer unit 14 and the temporary audio buffer unit 24,storage data items are successively read from the video buffer unit 13and the audio buffer unit 23. Then, as illustrated in FIG. 2A, the readstorage data items are subjected to the time synchronization processingby the time synchronization processing unit 15 and the format conversionprocessing by the format conversion processing unit 16 and recorded inthe video/audio/time file storage unit 26A as the video/audio/time file26.

If it is determined that the storage data amounts of the buffer memorieswhose storage capacity has temporarily exceeded 80% are reduced to lessthan a predetermined value as a result of the time synchronizationprocessing and the format conversion processing subjected to the storagedata items read from the video buffer unit 13 and the audio buffer unit23, the information processing device changes to the restoration mode.

Here, the restoration mode refers to a mode in which the digitized videodata items stored in the temporary video buffer unit 14 and thedigitized audio data items stored in the temporary audio buffer unit 24in the high load mode are subjected to the time synchronizationprocessing by the time synchronization processing unit 15 and the formatconversion processing by the format conversion processing 16 and storedin the video/audio/time file storage unit 26A as the video/audio/timefile 26, thereby processing the storage data items of the temporaryvideo buffer unit 14 and the temporary audio buffer unit 24.

Here, a case where the storage data amounts of the video buffer unit 13and the audio buffer unit 23 are reduced to less than the predeterminedvalue can be recognized, for example, as a case where the storage dataamounts of the respective buffer memories constituting the video bufferunit 13 and the audio buffer unit 23 become less than 80% of theirstorage capacities or to a case where the storage data items of thevideo buffer unit 13 and the audio buffer unit 23 are completelyprocessed.

In the restoration mode, as illustrated in FIG. 2C, the timesynchronization processing unit 15 and the format conversion processingunit 16 start applying the time synchronization processing and theformat conversion processing, respectively, to the digitized video dataitems recorded in the video data file 14 b, the time information itemscorresponding to the digitized video data items recorded in the videotime information file 14 a, the digitized audio data items recorded inthe audio data file 24 b, and the time information items correspondingto the digitized audio data items recorded in the audio time informationfile 24 a.

In the time synchronization processing by the time synchronizationprocessing unit 15 and the format conversion processing by the formatconversion processing unit 16, the video data file 14 b in which thedigitized video data items and the time information items correspondingto the video data items are separately recorded in order of time, thevideo time information file 14 a in which the video time informationitems corresponding to the digitized video data items are separatelyrecorded in order of time, the audio data file 24 b in which thedigitized audio data items are separately recorded in order of time, andthe audio time information file 24 a in which the audio time informationitems are separately recorded are rerecorded in chronological order togenerate the video/audio/time file 26.

The time synchronization processing by the time synchronizationprocessing unit 15 and the format conversion processing by the formatconversion processing unit 16 are described below with reference to FIG.3.

FIG. 3 is a diagram for illustrating the recorded information items ofthe video time information file 14 a, the video data file 14 b, theaudio time information file 24 a, and the audio data file 24 b andillustrating the recorded information items of the video/audio/time file26 obtained by applying the time synchronization processing and theformat conversion processing to the video time information file 14 a,the video data file 14 b, the audio time information file 24 a, and theaudio data file 24 b.

As illustrated in FIG. 3, in the video time information file 14 a, thetime information items indicating the times at which the video dataitems were recorded and the data sizes of the video data items recordedat the corresponding times are stored in order of time. In the videodata file 14 b, the corresponding video data items per se are stored inorder of time.

Similarly, in the audio time information file 24 a, the time informationitems indicating the times at which the audio data items were recordedand the data sizes of the audio data items recorded at the correspondingtimes are stored in order of time. In the audio data file 24 b, thecorresponding audio data items per se are stored in order of time.

In the time information section 26 a at the upper stage of thevideo/audio/time information file 26, the time information itemsindicating the times at which the video data items and the audio dataitems were recorded and the data sizes of the video data items and theaudio data items recorded at the corresponding times are stored in orderof time. In the video/audio data section 26 b at the lower stage of thevideo/audio/time information file 26, the corresponding video data itemsand the audio data items per se are alternately stored in order of time.

From among the video data file 14 b in which the video data items per seare separately recorded in order of time (i.e., times 0, 1, 2, . . . ,N) by the time synchronization processing and the format conversionprocessing and video time information file 14 a in which the timeinformation items and the data sizes are separately recorded in order oftime (i.e., times 0, 1, 2, . . . , N) by the time synchronizationprocessing and the format conversion processing, the video data itemsand the audio data items at the same time stored in the video data file14 b and the audio data file 24 b, respectively, are alternately storedin the video/audio data section 26 b of the video/audio/time file 26 inorder of time. In other words, in the video/audio data section 26 b ofthe video/audio/time file 26, the digitized video data item at time 0,the digitized audio data item at time 0, the digitized video data itemat time 1, the digitized audio data item at time 1, . . . , thedigitized video data item at time N, and the digitized audio data itemat time N are recorded in this order.

Similarly, in the time information section 26 a of the video/audio/timefile 26, the time information items at the same time stored in the videotime information file 14 a and the audio time information file 24 a,respectively, are alternately recorded in order of time. In other words,the time information item representing time 0, the data size of thevideo data item, and the data size of the audio data item each servingas the time information item at time 0, the time information itemrepresenting time 1, the data size of the audio data item, and the datasize of the audio data item each serving as the time information item attime 1, . . . , and the time information item representing time N, thedata size of the video data item, and the data size of the audio dataitem each serving as the time information item at time N are recorded inthis order in the time information section 26 a of the video/audio/timefile 26.

When moving images are actually reproduced from the video/audio/timefile 26 thus generated, the following processing is performed.

In other words, the time information items, the data sizes of the videodata items at the corresponding times, and the data sizes of the audiodata items at the corresponding times are successively read from thebeginning of the time information section 26 a of the video/audio/timefile 26 in order of time. Then, the video data items and the audio dataitems at the corresponding times are read from the video/audio datasection 26 b of the video/audio/time file 26 by amounts equivalent tothe data sizes of the specified video data items and the data sizes ofthe specified audio data items. Since the video data items and the audiodata items are alternately stored for each time in the video/audio datasection 26 b in order of time, the data items of corresponding dataamounts are successively read from the video/audio data section 26 b inaccordance with the data sizes of the video data items and the datasizes of the audio data items at the corresponding times stored in thetime information section 26 a. With this operation, the video data itemsand the audio data items at the corresponding times are successivelyalternately read and reproduced.

As described above, according to the embodiment of the presentinvention, in the high load mode, when respective data items arerecorded, the time information items are separated from the digitizedvideo data items and the digitized audio data items at everypredetermined time interval without the application of the timesynchronization processing and the format conversion processing to thedigitized video data items and the digitized audio data items. Then, thedigitized video data items, the time information items corresponding tothe digitized video data items, the digitized audio data items, and thetime information items corresponding to the digitized audio data itemsare separately recorded per se (the video time information file 14 a,the video data file 14 b, the audio time information file 24 a, and theaudio data file 24 b). In the restoration mode, the video timeinformation file 14 a, the video data file 14 b, the audio timeinformation file 24 a, and the audio data file 24 b are subjected to thetime synchronization processing and the format conversion processing soas to be combined with each other, thus generating the video/audio/timefile 26. In other words, if input data amounts exceed the processingperformance of the information processing device related to timesynchronization processing and the format conversion processing, theinformation processing device changes to the high load mode. In thismode, the information processing device leaves the application of thetime synchronization processing and the format conversion processing toinput data items on the back burner and temporarily stores the inputdata items.

Therefore, the input data items exceeding the processing performance ofthe information processing device related to the time synchronizationprocessing and the format conversion processing are not required to beabandoned. That is, the input data items can be reliably recorded.

Here, the data amounts per unit time of the digitized video data itemsand the digitized audio data items, i.e., bit frames, frame rates, etc.,are different. Therefore, when the digitized video data items and thedigitized audio data items are successively read from the video bufferunit 13 and the audio buffer unit 23, respectively, at a constant readspeed, read digitized video data items and read digitized audio dataitems are not necessarily synchronized with each other. Accordingly,time synchronization processing is required to synchronize the digitizedvideo data items and the digitized audio data items.

FIG. 5 is a flowchart for illustrating an operating procedure in thetime synchronization processing by the time synchronization processingunit 15 according to the embodiment of the present invention. Note thatthe time synchronization processing at the time of reading storage dataitems from the video buffer unit 13 and the audio buffer unit 23 and thetime synchronization processing at the time of reading storage dataitems from the temporary video buffer unit 14 and the temporary audiobuffer unit 24 are described below one by one.

First, the time synchronization processing at the time of reading thestorage data items from the video buffer unit 13 and the audio bufferunit 23 is described.

In step S51, the digitized video data item at time i (i=0 through N) isread from the video buffer unit 13 together with the time informationitem accompanied by time i.

Next, the digitized audio data item having the time information itemindicating time i the same as the time of the video data item acquiredfrom the video buffer unit 13 is searched for from the audio buffer unit23.

Generally, the data amounts per unit time of the video information itemsobtained through the camera 11 and the data amounts per unit time of theaudio information items obtained through the microphone 21 are differentfrom each other. Particularly, the data amounts per unit time of theaudio information items are smaller than the data amounts per unit timeof the video information items in many cases. Therefore, all thedigitized video data items do not necessarily have the correspondingdigitized audio data items. This is because, when the data amounts perunit time of the audio information items are relatively small, thegeneration frequency of the digitized audio data items becomes smallerthan that of the digitized video data items at the time of digitizinginput data items. Accordingly, the digitized audio data itemcorresponding to the digitized video data item at certain time may notexist. In this case, the flow proceeds to “No” in step S52.

If the flow proceeds to “No” in step S52, the digitized video data itemand the time information item corresponding to the digitized video dataitems read in step S51 are written in the video/audio/time file 26 instep S53 (i.e., the time information item, the data size of the videodata item, and the video data item at time i in the data structure onthe right side of FIG. 3).

If the corresponding digitized audio data item at time i is found (“Yes”in step S52), it is read from the audio buffer unit 23 together with thetime information item accompanied by the corresponding digitized audiodata item in step S54. Then, in step S55, the corresponding digitizedaudio data item and the time information item are written in thevideo/audio/time file 26 (i.e., the time information item, the data sizeof the audio data item, and the audio data item at time i in the datastructure on the right side of FIG. 3).

In step S56, a determination is made as to whether all the digitizedvideo data items of the video data file 14 have been processed (i.e.,whether the digitized video data item at time N has been processed).

If all the digitized video data items have been processed (“Yes” inS56), the time synchronization processing is completed. If all thedigitized video data items have not been processed (“No” in S56), theflow proceeds to step S57 where 1 is added to the time variable i. Then,the flow returns to step S51 to repeat the above processing. Thevideo/audio/time file 26 is thus generated.

Next, the time synchronization processing at the time of reading thestorage data items from the temporary video buffer unit 14 and thetemporary audio buffer unit 24 is described.

In step S51, the digitized video data item at time i (i=0 through N) isread from the video data file 14 b stored in the temporary video bufferunit 14, while the time information item and the data size of the videodata item each serving as the video time information item at the sametime i are read from the video time information file 14 a.

Then, in step S52, the audio time information item or the digitizedaudio data item at the same time is searched for from the audio timeinformation file 24 a or the audio data file 24 b stored in thetemporary audio buffer unit 24.

If the audio time information item or the digitized audio data item atthe same time i is not found (“No” in step S52), the digitized videodata item and the time information item acquired in step S51 are writtenin the video/audio/time file 26 (i.e., the time information item, thedata size of the video data item, and the video data item at time i inthe data structure on the right side of FIG. 3).

On the other hand, if the audio time information item or the digitizedaudio data item at the same time is found (“Yes” in step S52), thecorresponding audio time information item and the digitized audio dataitem are read from the audio time information file 24 a and the audiodata file 24 b, respectively, stored in the temporary audio buffer unit24 b in step S54. In step S55, the read audio time information item andthe digitized audio data item are written in the video/audio/time file26 (i.e., the time information, the data size of the audio data item,and the audio data item at time i in the data structure on the rightside of FIG. 3).

In step S56, a determination is made as to whether all the timeinformation items and all the digitized video data items in the videotime information file 14 and the video data file 14 b, respectively,stored in the video temporary buffer 14 have been processed (i.e.,whether the time information item and the digitized video data item attime N have been processed).

If all the time information items and all the digitized video data itemshave been processed (“Yes” in step S56), the time synchronizationprocessing is completed. If all the time information items and all thedigitized video data items have not been processed (“No” in step S56),the flow proceeds to step S57 where 1 is added to the time variable i.Then, the flow returns to step S51 to repeat the above processing. Thevideo/audio/time file 26 is thus generated.

Next, with reference to FIGS. 1 and 6 through 11, the flow of movingimage recording operations in the information processing device,particularly, a mode change operation between the low load mode, thehigh load mode, and the restoration mode is described.

FIG. 6 is an operations flowchart paying attention to the flow of themoving image recording operations in the information processing device,particularly, to the mode change operation.

When recording operations of moving images, i.e., photographing andrecording operations by the user, are started, video information itemsand audio time information items are output from the camera 11 and themicrophone 21, respectively. The output video information items and theaudio time information items are digitized by the video digitizationunit 12 and the audio digitization unit 22, respectively. Thus, thedigitized video data items accompanied by the time information items andthe digitized audio data items accompanied by the time information itemsare generated.

In step S2, a determination is made as to whether the storage dataamount of the video buffer unit 13 or the audio buffer unit 23 exceeds80% of its storage capacity.

If it is determined that the storage data amount has not exceeded 80% ofthe storage capacity (“No” in step S2), a determination is made as towhether the storage destinations of the present digitized video dataitems and the digitized audio data items are the temporary video bufferunit 14 and the temporary audio buffer unit 24 in step S3.

If it is determined that the storage destinations are not the temporaryvideo buffer unit 14 and the temporary audio buffer unit 24 (“No” instep S3), i.e., if it is determined that the storage destinations arethe video buffer unit 13 and the audio buffer unit 23, the digitizedvideo data items and the digitized audio data items are stored in thevideo buffer unit 13 and the audio buffer unit 23, respectively, so asto be accompanied by the corresponding time information items (step S4).

In step S5, a determination is made as to whether the storage dataamount of the video buffer unit 13 or the audio buffer unit 23 exceeds80% of its storage capacity.

If it is determined that the storage data amount does not exceed thestorage capacity (“No” in step S5), the time synchronization processingunit 15 and the data format conversion processing unit 16 of thecombining unit 25 apply the time synchronization processing and theformat conversion processing, respectively, to the digitized video dataitems accompanied by the time information items and the digitized audiodata items accompanied by the time information items stored in the videobuffer unit 13 and the audio buffer unit 23, thereby generating thevideo/audio/time file 26 (step S6).

The processing in steps S1 through S6 is repeatedly performed until itis determined that the storage data amount of the video buffer unit 13or the audio buffer unit 23 exceeds 80% of the storage capacity in stepS2 or step S5.

The above state refers to the low load mode.

As illustrated in FIGS. 7 and 8, in the low load mode, the digitizedvideo data items accompanied by the time information items and thedigitized audio data items accompanied by the time information items areaccumulated in the video buffer unit 13 and the audio buffer unit 23,respectively. At the same time, the processing by the timesynchronization processing unit 15 and the processing by the formatconversion processing unit 16 of the combining unit 25 are applied tothe digitized video data items and the digitized audio data items storedin the video buffer unit 13 and the audio buffer unit 23, respectively,to generate the video/audio/time file 26. The generated video/audio/timefile 26 is stored in the video/audio/time file storage unit 26A.

FIG. 7 illustrates a state in which the storage destinations of thedigitized video data items and the digitized audio data items outputfrom the video digitization unit 12 and the audio digitization unit 22,respectively, are changed from the video buffer unit 13 and the audiobuffer unit 23 to the temporary video buffer unit 14 and the temporaryaudio buffer unit 24 when the modes are changed from the low load modeto the restoration mode through the high load mode. Furthermore, FIG. 7illustrates a state in which the storage places of the data items to beprocessed when the digitized video data items and the digitized audiodata items thus stored are processed by the combining unit 25 arechanged from the video buffer unit 13 and the audio buffer unit 23 tothe temporary video buffer unit 14 and the temporary audio buffer unit24.

FIG. 8 is a diagram for illustrating the operations of the informationprocessing device along with FIG. 1.

As illustrated in times t1 through t4 in FIG. 11, when a processingload, i.e., the data amounts per unit time of the digitized video dataitems and the digitized audio data items output from the videodigitization unit 12 and the audio digitization unit 22, respectively,gradually increase, the processing by the time synchronizationprocessing unit 15 and the processing by the format conversionprocessing unit 16 are not properly processed in time. As a result, thestorage data amounts of the video buffer unit 13 and the audio bufferunit 23 gradually increase.

FIG. 9 is a diagram for illustrating the operations of the informationprocessing device in FIG. 1 in the high load mode. FIG. 11 is a diagramfor illustrating the mode change between the low load mode, the highload mode, and the restoration mode along with the change of theprocessing load in the information processing device in FIG. 1.

If the storage data amount of the video buffer unit 13 or the audiobuffer unit 23 exceeds 80% of its storage capacity (time t4 in FIG. 11),the flow in FIG. 6 proceeds to “Yes” in step S2, and the storagedestinations of the digitized video data items and the digitized audiodata items are changed from the video buffer unit 13 and the audiobuffer unit 23 to the temporary video buffer unit 14 and the temporaryaudio buffer unit 24 in step S8.

Also, as the flow of another operation, the flow proceeds to “Yes” instep S5 in FIG. 6, and the storage destinations of the digitized videodata items and the digitized audio data items are changed from the videobuffer unit 13 and the audio buffer unit 23 to the temporary videobuffer unit 14 and the temporary audio buffer unit 24 in step S7. At thesame time, in step S6, the time synchronization processing by the timesynchronization processing unit 15 and the format conversion processingby the format conversion processing unit 16 are continuously applied tothe digitized video data items accompanied by the time information itemsand the digitized audio data items accompanied by the time informationitems stored in the video buffer unit 13 and the audio buffer unit 23,respectively.

In other words, if it is determined that the flow proceeds to “Yes” instep S2 or S5 in FIG. 6, the information processing device changes tothe high load mode.

As illustrated in FIGS. 7 and 9, in the high load mode, the digitizedvideo data items accompanied by the time information items and thedigitized audio data items accompanied by the time information itemsoutput from the video digitization unit 12 and the audio digitizationunit 22, respectively, are temporarily stored in the temporary videobuffer unit 14 and the temporary audio buffer unit 24, respectively.

At that time, the time information items are separated from thedigitized video data items accompanied by the time information items andthe digitized audio data items accompanied by the time informationitems. That is, the video time information file 14 a, the video datafile 14 b, the audio time information file 24 a, and the audio data file24 b are separately recorded (see FIGS. 2B, 4A, and 4B).

At the same time, the time synchronization processing by the timesynchronization processing unit 15 and the format conversion processingby the format conversion processing unit 16 are continuously applied tothe digitized video data items accompanied by the time information itemsand the digitized audio data items accompanied by the time informationitems stored in the video buffer unit 13 and the audio buffer unit 23,respectively.

Thus, the digitized video data items, the digitized audio data items,and the time information items corresponding to the digitized video dataitems and the digitized audio data items are stored in thevideo/audio/time file storage unit 26A as the video/audio/time file 26as in the case of the low load mode.

As illustrated in FIGS. 7 and 9, in the high load mode, the storagedestinations of the digitized video data items and the digitized audiodata items are changed from the video buffer unit 13 and the audiobuffer unit 23 to the temporary video buffer unit 14 and the temporaryaudio buffer unit 24. Therefore, the storage data items in the videobuffer unit 13 and the audio buffer unit 23 no longer increase, whilethe data items stored in the video buffer unit 13 and the audio bufferunit 23 are gradually reduced by the processing of the combining unit 25(times t4 through t7 in FIG. 11).

Referring back to FIG. 6, when the flow proceeds to “Yes” in step S2 andthen the storage destinations of the digitized video data items and thedigitized audio data items are changed from the video buffer unit 13 andthe audio buffer unit 23 to the temporary video buffer unit 14 and thetemporary audio buffer unit 24 in step S8, a determination is made as towhether all the storage data items in the video buffer unit 13 and theaudio buffer unit 23 have been processed in step S9.

As described above, in the high load mode, while the digitized videodata items, the digitized audio data items, and the time informationitems corresponding to the digitized video data items and the digitizedaudio data items are stored in the temporary video buffer unit 14 andthe temporary audio buffer unit 24, respectively, the timesynchronization processing by the time synchronization processing unit15 and the format conversion processing unit by the format conversionprocessing unit 16 are applied to the digitized video data itemsaccompanied by the time information items and the digitized audio dataitems accompanied by the time information items stored in the videobuffer unit 13 and the audio buffer unit 23, respectively. As thestorage data items of the video buffer unit 13 and the audio buffer unit23 are successively processed as described above, the storage dataamounts (i.e., usage rates of the buffers) of the video buffer unit 13and the audio buffer unit 23 gradually reduce as illustrated in times t4through t7 in FIG. 11. As a result, if all the storage data items of thevideo buffer unit 13 and the audio buffer unit 23 have been processed,it is determined that the flow proceeds to “Yes” in step S9.

Note that the criterion of determining the flow in step S9 is notlimited to the “time at which all the storage data items of the videobuffer unit 13 and the audio buffer unit 23 have been processed,” but itmay be based on the “time at which the storage data amounts of the videobuffer unit 13 and the audio buffer unit 23 have been reduced to apredetermined threshold (e.g., 80% of the storage capacity).”

If it is determined that the flow proceeds to “No” in step S9, i.e., ifthe storage data items still remain in the video buffer unit 13 and theaudio buffer unit 23, the flow returns to step S1. Then, the processingin steps S1, S2, S8, and S9 is repeatedly performed until it isdetermined that the flow proceeds to “Yes” in step S9.

Furthermore, as the flow of another operation to change to the high loadmode, the flow proceeds to “Yes” in step S5 as described above. Inaddition, in step S7, the storage destinations of the digitized videodata items and the digitized audio data items are changed from the videobuffer unit 13 and the audio buffer unit 23 to the temporary videobuffer unit 14 and the temporary audio buffer unit 24. Moreover, in stepS6, the time synchronization processing by the time synchronizationprocessing unit 15 and the format conversion processing by the formatconversion processing unit 16 are continuously applied to the digitizedvideo data items accompanied by the time information items and thedigitized audio data items accompanied by the time information itemsstored in the video buffer unit 13 and the audio buffer unit 23,respectively.

In this case, the flow returns to step S1. Then, the flow proceeds tostep S2, and it is determined that the flow proceeds to “Yes” in stepS2. In addition, in step S8, the storage destinations of the digitizedvideo data items and the digitized audio data items are changed from thevideo buffer unit 13 and the audio buffer unit 23 to the temporary videobuffer unit 14 and the temporary audio buffer unit 24. The followingprocessing is the same as the processing described above. The processingin steps S1, S2, S8, and S9 is repeatedly performed until it isdetermined that the flow proceeds to “Yes” in step S9

On the other hand, if it is determined that the flow proceeds to “Yes”in step S9, i.e., if all the storage data items in the video buffer unit13 and the audio buffer unit 23 have been processed by the timesynchronization processing by the time synchronization processing unit15 and the format conversion processing by the format conversionprocessing unit 16, i.e., if the state at time t8 is established in FIG.11, the digitized video data items accompanied by the time informationitems and the digitized audio data items accompanied by the timeinformation items output from the video digitization unit 12 and theaudio digitization unit 22, respectively, are stored in the video bufferunit 13 and the audio buffer unit 23, respectively, so as to beaccompanied by the corresponding time information items in step S10.

In step S11, the time synchronization processing by the timesynchronization processing unit 15 and the format conversion processingby the format conversion processing unit 16 are applied to the videotime information file 14 a, the video data file 14 b, the audio timeinformation file 24 a, and the audio data file 24 b separately stored inthe temporary video buffer unit 14 and the temporary audio buffer unit24 as described above.

This state refers to the restoration mode as illustrated in FIG. 2C.

FIG. 10 is a diagram for illustrating the flow of processing data itemsin the restoration mode.

In the restoration mode, as in the case of the low load mode, thedigitized video data items accompanied by the time information items andthe digitized audio data items accompanied by the time information itemsoutput from the video digitization unit 12 and the audio digitizationunit 22, respectively, are stored in the video buffer unit 13 and theaudio buffer unit 23 so as to be accompanied by the time informationitems, while the time synchronization processing by the timesynchronization processing unit 15 and the format conversion processingby the format conversion processing unit 16 are applied to the videotime information file 14 a, the video data file 14 b, the audio timeinformation file 24 a, and the audio data file 24 b separately stored inthe temporary video buffer unit 14 and the temporary audio buffer unit24 (see FIG. 7).

In the restoration mode, the storage destinations of the digitized videodata items and the digitized audio data items output from the videodigitization unit 12 and the audio digitization unit 22, respectively,are changed to the video buffer unit 13 and the audio buffer unit 23.Therefore, as illustrated in times t8 and t9 in FIG. 11, the storagedata amounts of the temporary video buffer unit 14 and the temporaryaudio buffer unit 24 are gradually reduced by the processing of thestorage data items of the temporary video buffer unit 14 and thetemporary audio buffer unit 24 through the time synchronizationprocessing unit 15 and the format conversion processing unit 16 of thecombining unit 25.

At that time, the storage data amounts of the video buffer unit 13 andthe audio buffer unit 23 are gradually increased when the digitizedvideo data items accompanied by the time information items and thedigitized audio data items accompanied by the time information itemsoutput from the video digitization unit 12 and the audio digitizationunit 22, respectively, are stored as described above.

Referring back to FIG. 6, a determination is made as to whether all thestorage data items of the temporary video buffer unit 14 and thetemporary audio buffer unit 24 have been processed by the timesynchronization processing unit 15 and the format conversion processingunit 16 of the combining unit 25 in step S12.

As described above, the storage data amounts of the temporary videobuffer unit 14 and the temporary audio buffer unit 24 are graduallyreduced when the storage data items of the temporary video buffer unit14 and the temporary audio buffer unit 24 are processed by the timesynchronization processing unit 15 and the format conversion processingunit 16 of the combining unit 25. As a result, if the storage dataamounts of the temporary video buffer unit 14 and the temporary audiobuffer unit 24 become zero, the flow proceeds to “Yes” in step S12.

Note that the criterion of determining the flow in step S12 is notlimited to the “time at which the storage data amounts of the temporaryvideo buffer unit 14 and the temporary audio buffer unit 24 become zero”as described above, but it may be based on the “time at which thestorage data amounts of the temporary video buffer unit 14 and thetemporary audio buffer unit 24 are reduced to a predetermined threshold(e.g., 80% of the storage capacity).”

The processing in steps S11 and S12 is repeatedly performed until thestorage data amounts of the temporary video buffer unit 14 and thetemporary audio buffer unit 24 become zero and it is determined that theflow proceeds to “Yes” in step S12, whereby the restoration mode ismaintained.

If the storage data amounts of the temporary video buffer unit 14 andthe temporary audio buffer unit 24 become zero and it is determined thatthe flow proceeds to “Yes” in step S12, the flow proceeds to step S6.Then, as illustrated in FIG. 7, objects to be processed by the timesynchronization processing unit 15 and the format conversion processingunit 16 of the combining unit 25 are changed to the digitized video dataitems accompanied by the time information items and the digitized audiodata items accompanied by the time information items stored in the videobuffer unit 13 and the audio buffer unit 23, respectively, as describedabove.

As a result, the information processing device returns to the low loadmode based on FIG. 8.

As illustrated in FIG. 11, if the processing load, i.e., the dataamounts per unit time of the digitized video data items and thedigitized audio data items output from the video digitization unit 12and the audio digitization unit 22, respectively, are reduced betweenthe high load mode at the times t1 through t9 and the followingrestoration mode, the information processing device returns to therestoration mode in FIG. 8. Then, as described above, the low load modeis maintained until it is determined that the flow proceeds to “Yes” instep S2 or S5 in accordance with the increase in the processing load.

On the other hand, if the processing load, i.e., the data amounts perunit time of the digitized video data items and the digitized audio dataitems output from the video digitization unit 12 and the audiodigitization unit 22, respectively, are not reduced but remain at highlevel between the high load mode at the times t1 through t9 and thefollowing restoration mode, the information processing device returns tothe low load mode as illustrated in FIG. 8. Then, when the storage dataitems of the video buffer unit 13 and the audio buffer unit 23 are notproperly processed in time by the time synchronization processing unit15 and the format conversion processing unit 16 of the combining unit25, the storage data amounts of the video buffer unit 13 and the audiobuffer unit 23 exceed 80% of the storage capacity and it is determinedthat the flow proceeds to “Yes” in step S2 or S5. Then, the informationprocessing device changes to the high load mode in FIG. 9. Subsequently,the processing of “high load mode→restoration mode→low load mode→highload mode→ . . . ” is repeatedly performed until the processing load isreduced as illustrated in FIG. 7.

According to the embodiment of the present invention, the followingadvantages are obtained.

In other words, when audio information items and time information itemsare recorded, it is required to perform the time synchronizationprocessing and the format conversion processing that search fordigitized video data items and digitized audio data items at the sametime to ensure synchronization between them so as to simultaneouslyrecord the digitized video data items and the digitized audio data itemsin the same file. In this case, with an improvement in the quality ofvideos and audios to be recorded, the time synchronization processingand the format conversion processing require a certain processing time,which may result in a limitation in processing performance.

According to the embodiment of the present invention, in the high loadmode, the digitized video data items and the digitized audio data itemsare separately stored in the different files, and the time informationitems corresponding to the digitized video data items and the digitizedaudio data items are stored in the different files. In other words, thedigitized video data items and the digitized audio data items aresuccessively recorded without being subjected to the timesynchronization processing by which the digitized video data items andthe digitized audio data items are synchronized with each other.

As a result, even if the improvement in the quality of videos and audiosto be recorded is achieved, it is possible to reliably record input dataitems regardless of the processing performance of a device related tothe time synchronization processing.

Furthermore, according to the embodiment of the present invention, thedigitized video data items and the digitized audio data items separatelyrecorded in the high load mode are read and subjected to the timesynchronization processing and the format conversion processing. As aresult, it is possible to store the video/audio/time file in a specifiedformat in a recording medium.

FIG. 12 is a block diagram illustrating a configuration example of acomputer for describing a case in which the information processingdevice according to the embodiment of the present invention isimplemented by the computer.

As illustrated in FIG. 12, the computer 500 has a CPU 501 that issuesinstructions constituting specified programs to execute variousoperations; an operations unit 502 that is composed of a keyboard, amouse, and the like and through which the user inputs operating contentsor data items; a display unit 503 that is composed of a CRT, a liquidcrystal display, or the like and displays processing processes,processing results, or the like by the CPU 501 to the user; a memory 504that is composed of a ROM, a RAM, and the like and that stores programs,data items, and the like executed by the CPU 501 and serves as a workarea; a hard disk unit 505 that stores programs, data items, and thelike; a CD-ROM drive 506 that loads programs and data items from theoutside through a CD-ROM 507 serving as a medium; and a modem 508 thatdownloads programs from outside servers via a communication network 509such as the Internet and a LAN.

In addition, the computer 500 has the camera 11, the microphone 21, andthe video digitization unit 12 and the audio digitization unit 22 thatdigitize the video information items and the audio information items,respectively, output from the camera 11 and the microphone 21 togenerate the digitized video data items and the digitized audio dataitems.

The computer 500 loads or downloads the program having instructions forcausing the CPU 501 to execute the processing by the informationprocessing device according to the embodiment of the present inventionthrough the CD-ROM 507 serving as a medium or through the communicationnetwork 509 serving as a medium. Then, the computer 500 installs theloaded or downloaded program in the hard disk unit 505 and appropriatelyloads the same in the memory 504 so as to be executed by the CPU 501. Asa result, the computer 500 implements the information processing deviceaccording to the embodiment of the present invention.

All examples and conditional language recited herein are intended forpedagogical purposes to aid the reader in understanding the presentinvention and the concepts contributed by the inventor to furthering theart, and are to be construed as being without limitation to suchspecifically recited examples and conditions, and the organization ofsuch examples in the specification does not relate to a showing of thesuperiority or inferiority of the present invention. Although theembodiment of the present invention has been described in detail, itshould be understood that the various changes, substitutions, andalterations could be made hereto without departing from the spirit andscope of the present invention.

What is claimed is:
 1. A moving image recording method comprising:digitizing a video information item obtained by photographing and anaudio information item obtained by recording to obtain a digitized videodata item and a digitized audio data item each accompanied by a timeinformation item, respectively; separating the digitized video data itemaccompanied by the time information item into the time information itemand the digitized video data item and temporarily storing the timeinformation item and the digitized video data item separately;separating the digitized audio data item accompanied by the timeinformation item into the time information item and the digitized audiodata item and temporarily storing the time information item and thedigitized audio data item separately; and combining the time informationitem and the digitized video data item that are separately stored in thetemporary storing the time information and the digitized video data itemwith the time information item and the digitized audio data item thatare separately stored in the temporarily storing the time informationand the digitized audio data item in synchronization with each other soas to store the time information items, the digitized video data item,and the digitized audio data item in a predetermined format.
 2. Themoving image recording method according to claim 1, having a firstpredetermined mode, a second predetermined mode and a thirdpredetermined mode, wherein the first predetermined mode executesprocessing of combining the digitized video data item obtained in thedigitizing and accompanied by the time information item with thedigitized audio data item obtained in the digitizing and accompanied bythe time information item in synchronization with each other and storingthe digitized video data item accompanied by the time information itemand the digitized audio data item accompanied by the time informationitem in the predetermined format, the second predetermined mode executesprocessing of separating the digitized video data item obtained in thedigitizing and accompanied by the time information item into the timeinformation item and the digitized video data item and temporarilystoring the time information item and the digitized video data itemseparately in the separating and temporarily storing the timeinformation item and the digitized video data item and separating thedigitized audio data item obtained in the digitizing and accompanied bythe time information item into the time information item and thedigitized audio data item and temporarily storing the time informationitem and the digitized audio data item separately in the separating andtemporarily storing the time information item and the digitized audiodata item, and the third predetermined mode executes processing ofcombining the time information item and digitized video data item thatare separately stored in the temporarily storing the time informationitem and the digitized video data item with the time information itemand the digitized audio data item that are separately stored in thetemporarily storing the time information item and the digitized audiodata item in synchronization with each other and storing the timeinformation items, the digitized video data item, and the digitizedaudio data item.
 3. The moving image recording method according to claim2, wherein buffers are provided to temporarily store the digitized videodata item obtained in the digitizing and accompanied by the timeinformation item and the digitized audio data item obtained in thedigitizing and accompanied by the time information item, respectively,the first predetermined mode executes the processing of reading thedigitized video data item stored in the corresponding buffer andaccompanied by the time information item and the digitized audio dataitem stored in the corresponding buffer and accompanied by the timeinformation item, combining the digitized video data item accompanied bythe time information item and the digitized audio data item accompaniedby the time information item in synchronization with each other, andstoring the digitized video data item accompanied by the timeinformation item and the digitized audio data item accompanied by thetime information item in the predetermined format, the firstpredetermined mode changes to the second predetermined mode when datastorage amounts of the buffers reach a first predetermined value, thesecond predetermined mode changes to the third predetermined mode whenthe data storage amounts of the buffers reach a second predeterminedvalue smaller than the first predetermined value by processing in thefirst predetermined mode, and the third predetermined mode changes tothe first predetermined mode when a storage amount in the temporarilystoring the digitized video data item and the temporarily storing thedigitized audio data item reaches a third predetermined value.
 4. Themoving image recording method according to claim 2, wherein buffers areprovided to temporarily store the digitized video data item obtained inthe digitizing and accompanied by the time information item and thedigitized audio data item obtained in the digitizing and accompanied bythe time information item, respectively, the first predetermined modeexecutes the processing of reading the digitized video data itemaccompanied by the time information item and the digitized audio dataitem accompanied by the time information item from the correspondingbuffers, combining the digitized video data item accompanied by the timeinformation item and the digitized audio data item accompanied by thetime information item in synchronization with each other, and storingthe digitized video data item accompanied by the time information itemand the digitized audio data item accompanied by the time informationitem in the predetermined format, simultaneously with the digitizing ofthe video information item obtained by the photographing and the audioinformation item obtained by the recording to obtain the digitized videodata item accompanied by the time information item and the digitizedaudio data item accompanied by the time information item, respectively,the second predetermined mode executes the processing of reading thedigitized video data item accompanied by the time information item andthe digitized audio data item accompanied by the time information itemfrom the corresponding buffers, combining the digitized video data itemaccompanied by the time information item and the digitized audio dataitem accompanied by the time information item in synchronization witheach other, and storing the digitized video data item accompanied by thetime information item and the digitized audio data item accompanied bythe time information item in the predetermined format, simultaneouslywith the separating and temporarily storing the time information itemand the digitized video data item of separating the digitized video dataitem obtained in the digitization and accompanied by the timeinformation item into the time information item and the digitized videodata item and temporarily storing the time information item and thedigitized video data item separately and simultaneously with theseparating and temporarily storing the time information item and thedigitized audio data item of separating the digitized audio data itemobtained in the digitizing and accompanied by the time information iteminto the time information item and the digitized audio data item andtemporarily storing the time information item and the digitized audiodata item separately, and the third predetermined mode executes theprocessing of combining the time information item and the digitizedvideo data item that are separately stored in the temporarily storingthe time information item and the digitized video data item with thetime information item and the digitized audio data item that areseparately stored in the temporarily storing the time information itemand the digitized audio data item in synchronization with each other andstoring the time information items, the digitized video data item, andthe digitized audio data item, simultaneously with the digitizing of thevideo information item obtained by the photographing and the audioinformation item obtained by the recording to obtain the digitized videodata item accompanied by the time information item and the digitizedaudio data item accompanied by the time information item, respectively.5. The moving image recording method according to claim 1, wherein inthe digitizing, the video information item obtained by the photographingand the audio information item obtained by the recording are sorted atevery predetermined time interval to generate the digitized video dataitem and the digitized audio data item at every predetermined timeinterval, respectively, and the time information item is composed of aninformation item on a data size of the corresponding digitized videodata item and the corresponding digitized audio data item at everypredetermined time interval, in the temporarily storing the timeinformation item and the digitized video data item, the digitized videodata item at every predetermined time interval is recorded in order oftime and the time information item corresponding to the digitized videodata item is recorded in order of time, in the temporarily storing thetime information item and the digitized audio data item, the digitizedaudio data item at every predetermined time interval is recorded inorder of time and the time information item corresponding to thedigitized audio data item is recorded in order of time, and in thecombining, the digitized video data item at every predetermined timeinterval temporarily stored in the temporarily storing the timeinformation item and the digitized video data item and the digitizedaudio data item at every predetermined time interval temporarily storedin the temporarily storing the time information item and the digitizedaudio data item are successively alternately recorded at everycorresponding time interval, and the time information itemscorresponding to the digitized video data item and the time informationitem corresponding to the digitized audio data item are successivelyalternately recorded at every corresponding time interval.
 6. Aninformation processing device comprising: a digitization unit thatdigitizes a video information item obtained by photographing and anaudio information item obtained by recording to obtain a digitized videodata item and a digitized audio data item each accompanied by a timeinformation item, respectively; a temporary video data item storage unitthat separates the digitized video data item accompanied by the timeinformation item into the time information item and the digitized videodata item and temporarily stores the time information item and thedigitized video data item separately; a temporary audio data itemstorage unit that separates the digitized audio data item accompanied bythe time information item into the time information item and thedigitized audio data item and temporarily stores the time informationitem and the digitized audio data item separately; and atime-information-item-separated format conversion unit that combines thetime information item and the digitized video data item that areseparately stored by the temporary video data item storage unit with thetime information item and the digitized audio data item that areseparately stored by the temporary audio data item storage unit insynchronization with each other and stores the time information items,the digitized video data item, and the digitized audio data item in apredetermined format.
 7. The information processing device according toclaim 6, further comprising a time-information-item-unseparated formatconversion unit that combines the digitized video data item obtained bythe digitization unit and accompanied by the time information item withthe digitized audio data item obtained by the digitization unit andaccompanied by the time information item in synchronization with eachother and stores the time information items, the digitized video dataitem, and the digitized audio data item in the predetermined format,wherein in a first predetermined mode, thetime-information-item-unseparated format conversion unit combines thedigitized video data item obtained by the digitization unit andaccompanied by the time information item with the digitized audio dataitem obtained by the digitization unit and accompanied by the timeinformation item in synchronization with each other and stores the timeinformation items, the digitized video data item, and the audio dataitem in the predetermined format, in a second predetermined mode, thetemporary video data item storage unit separates the digitized videodata item obtained by the digitization unit and accompanied by the timeinformation item into the time information item and the digitized videodata item and temporarily stores the time information item and thedigitized video data item separately, and the temporary audio data itemstorage unit separates the digitized audio data item obtained by thedigitization unit and accompanied by the time information item into thetime information item and the digitized audio data item and temporarilystores the time information item and the digitized audio data itemseparately, and in a third predetermined mode, thetime-information-item-separated format conversion unit combines the timeinformation item and digitized video data item that are separatelystored by the temporary video data item storage unit with the timeinformation item and the digitized audio data item that are separatelystored by the temporary audio data item storage unit in synchronizationwith each other and stores the time information items, the temporaryvideo data item, and the temporary audio data item.
 8. The informationprocessing device according to claim 7, further comprising buffersprovided to temporarily store the digitized video data item obtained bythe digitization unit and accompanied by the time information item andthe digitized audio data item obtained by the digitization unit andaccompanied by the time information item, respectively, wherein in thefirst predetermined mode, the time-information-item-unseparated formatconversion unit reads the digitized video data item stored in thecorresponding buffer and accompanied by the time information item andthe digitized audio data item stored in the corresponding buffer andaccompanied by the time information item, combines the digitized videodata item accompanied by the time information item and the digitizedaudio data item accompanied by the time information item insynchronization with each other, and stores the digitized video dataitem accompanied by the time information item and the digitized audiodata item accompanied by the time information item in the predeterminedformat, the first predetermined mode changes to the second predeterminedmode when data storage amounts of the buffers reach a firstpredetermined value, the second predetermined mode changes to the thirdpredetermined mode when the data storage amounts of the buffers reach asecond predetermined value smaller than the first predetermined value byprocessing of the time-information-item-unseparated format conversionunit, and the third predetermined mode changes to the firstpredetermined mode when a storage amount of the temporary video dataitem storage unit and the temporary audio data item storage unit reachesa third predetermined value by processing of thetime-information-item-separated format conversion unit.
 9. Theinformation processing device according to claim 7, wherein buffersprovided to temporarily store the digitized video data item obtained inthe digitization unit and accompanied by the time information item andthe digitized audio data item obtained in the digitization unit andaccompanied by the time information item, respectively, in the firstpredetermined mode, an operation by thetime-information-item-unseparated format conversion unit is executed, inwhich the digitized video data item accompanied by the time informationitem and the digitized audio data item accompanied by the timeinformation item are read from the corresponding buffers, combined insynchronization with each other, and stored in the predetermined format,simultaneously with an operation by the digitization unit in which thevideo information item obtained by the photographing and the audioinformation item obtained by the recording are digitized to obtain thedigitized video data item accompanied by the time information item andthe digitized audio data item accompanied by the time information item,respectively, in the second predetermined mode, an operation by thetime-information-item-unseparated format conversion unit is executed, inwhich the digitized video data item accompanied by the time informationitem and the digitized audio data item accompanied by the timeinformation item are read from the corresponding buffers, combined insynchronization with each other, and stored in the predetermined format,simultaneously with an operation by the temporary video data storageunit in which the digitized video data item obtained by the digitizationunit and accompanied by the time information item is separated into thetime information item and the digitized video data item and the timeinformation item and the digitized video data item are temporarilystored separately, and simultaneously with an operation by the temporaryaudio data storage unit in which the digitized audio data item obtainedby the digitization unit and accompanied by the time information item isseparated into the time information item and the digitized audio dataitem and the time information item and the digitized audio data item aretemporarily stored separately, and in the third predetermined mode, anoperation by the time-information-item-separated format conversion unitis executed, in which the time information item and the digitized videodata item that are separately stored by the temporary video data storageunit and the time information item and the digitized audio data itemthat are separately stored by the temporary audio data storage unit arecombined in synchronization with each other and stored in thepredetermined format, simultaneously with an operation by thedigitization unit in which the video information item obtained by thephotographing and the audio information item obtained by the recordingare digitized to obtain the digitized video data item accompanied by thetime information item and the digitized audio data item accompanied bythe time information item, respectively.
 10. The information processingdevice according to claim 6, wherein the digitization unit sorts thevideo information item obtained by the photographing and the audioinformation item obtained by the recording at every predetermined timeinterval to generate the digitized video data item and the digitizedaudio data item at every predetermined time interval, respectively, andthe time information item is composed of an information item on a datasize of the digitized video data item and the digitized audio data itemat every predetermined time interval, the temporary video data itemstorage unit records the digitized video data item at everypredetermined time interval in order of time and records the timeinformation item corresponding to the digitized video data item in orderof time, the temporary audio data item storage unit records thedigitized audio data item at every predetermined time interval in orderof time and records the time information item corresponding to thedigitized audio data item in order of time, and thetime-information-item-separated format conversion unit successivelyalternately records the digitized video data item at every predeterminedtime interval temporarily stored by the temporary video data itemstorage unit and the digitized audio data item at every predeterminedtime interval temporarily stored by the temporary audio data itemstorage unit at every corresponding time interval, and successivelyalternately records the time information item corresponding to thedigitized video data item and the time information item corresponding tothe digitized audio data item at every corresponding time interval.